Characterisation of a Stereo Matching and Object Location System

In this report we demonstrate and characterise a 3D wireframe model matcher, applied to a stereo object location problem typical of many industrial machine vision applications, and relate its design to the methodology we proposed as an earlier component of this project [41]. We begin by outlining the stereo requirements of a 3D model matcher, and in this context discuss the general characteristics of two approaches to solving the stereo correspondence problem: Feature Based Stereo (FBS) and Area Based Stereo (ABS). We show that FBS has significant advantages in terms of the accuracy of the recovered stereo data and of reduced susceptibility to image variations, but that ABS generally has a simpler flow of control and is therefore suitable for parallel processor or hardware implementations. We go on to describe the FBS and ABS algorithms implemented in our TINA vision system: PMF, a classical feature-based solution, and the Stretch Correlator (SC), a re-formulation of the algorithmic constraints of PMF into a feature driven, but fundamentally area-based, solution. We present a theoretical analysis of the errors involved in stereo calculations, and use it to show how the results from both stereo algorithms can be characterised in terms of correspondence mismatch and 3D reconstruction error. This leads to the conclusion that geometric fitting to 3D data should be done in disparity space, to exploit the property of isotropic errors. We also summarise an analysis of the mismatch probability of a stereo correspondence algorithm, and show that mismatches are inevitable and must be accounted for in later algorithm stages. We discuss the consequences of our analysis for the design of practical, modular 3D vision systems and describe the components of TINA relevant to this project. We then provide the results of evaluations of the performance of the TINA stereo algorithms, based on our analyses. In particular, we have found that SC has a mismatch probability of typically 1%-2% of the stereo data and in the worst case up to 8%. Finally, we give a practical demonstration of TINA’s 3D Wireframe Model Matcher, using both PMF and SC to reconstruct the 3D structure from stereo views of a brake shoe assembly and then locating a wireframe model of the assembly in the recovered 3D scenes. This is done with the object placed in a variety of orientations in a general scene with no special lighting. For similar object views, the number of features recovered by PMF and SC is shown to be identical, within statistical limits. For a set of 33 views of the object in different orientations, PMF provides matches in 18 cases and SC in 17 (rising to 21 and 20 matches respectively when the Canny double thresholds are reduced). A breakdown into sub-groups with similar object orientations shows that the numbers of matches remain identical within statistical limits in each sub-group. On the basis of these results, we conclude that our methodology can successfully be used to design and incrementally refine a modular vision system capable of stereo matching and 3D object location for “difficult” stereo reconstruction tasks. Moreover, the performance of our PMF and SC algorithms is to all intents and purposes identical, which given the computational simplicity of the latter may have positive implications for developing parallel or hardware implementations suitable for temporal stereo tasks.